METHOD AND APPARATUS FOR PROVIDING PROJECTED USER 



INTERFACE FOR COMPUTING DEVICE 



Field of the Invention 

The present invention is related to apparatus and techniques for projecting user 
5 interfaces for computing devices such as pervasive computing devices and, more 
particularly, for projecting a user input display and a user output display. 

Background of the Invention 

Pervasive computing devices such as PDAs (personal digital assistants), pocket 
PCs (personal computers), tablet PCs, and smart camera/PDA phones have steadily 

10 increased in popularity. Chip technology has made it possible for these devices to be 
small but functionally powerful. However, due to the decreased size of the devices, there 
is little to no room for a keyboard, and most include a tiny display. 

The addition of a virtual keyboard image for input and/or a virtual display for user 
output has been proposed as a way to attempt to improve the usability and portability of 

15 these pervasive computing devices. For example, U.S. Patent No. 6,323,942 discloses a 
virtual keyboard. Further, U.S. Patent No. 6,266,048 discloses a virtual display/keyboard 
for a PDA, which provides a solution to the limitations caused by the size of the 
pervasive computing device. A keyboard image is projected on a surface close to the user 
and a display image is projected on the same surface between the virtual keyboard image 

20 and the projecting device. While this increases the size of both the keyboard image and 
the display, the placement of both images on the same surface is uncomfortable for the 
user because the user must look downward to read the user output display. This is not the 
customary placement for a computer screen and a user would have to become accustomed 
to the new placement. The system of U.S. Patent No. 6,266,048 also uses a first projector 

25 for the keyboard image, and a second projector for the display. 
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The above mentioned approaches fail to provide a system that produces an 
optimal ergonomic and economic virtual input and output. Thus, a need exists for an 
improved virtual input and output system which overcomes these and other limitations. 

Summary of the Invention 

5 The present invention provides apparatus and techniques for projecting user 

interfaces for computing devices such as pervasive computing devices and, more 
particularly, for projecting a user input display and a user output display. 

For example, in one aspect of the invention, a technique of providing a user 
interface for a computing device comprises the following steps. A user input display is 

10 projected from a projector of a computing device on a first surface, while a user output 
display is projected from a projector of a computing device on a second surface. The first 
surface and the second surface are disposed in different planes. 

Advantageously, the technique enables the projection of the user input display on 
a separate surface from the user output display. This is accomplished through the use of a 

15 mirror system in the computing device which splits a single projected image into the user 
input display and the user output display. The user input display may be projected on a 
flat horizontal surface between the user and the computing device, while the user output 
display may be projected on a vertical surface behind the computing device. This allows 
a user to remain in a comfortable and customary position when the computing device is 

20 used, while maintaining the benefit of increased size of both the input and output 
displays. 

In a second aspect of the invention, a technique of providing a user interface for a 
computing device comprises the following steps. A user input display is projected from a 
projector of a computing device onto a surface, while a user output display is projected 
25 from the projector onto the same surface. A single projector is used for both the user 
input display and the user output display. Additionally, typing feedback may be emitted 



YOR920030103US1 



2 



when a keystroke is detected on the user input display or virtual keyboard image. This 
feedback may be in an audio and/or a visual format. 

These and other objects, features, and advantages of the present invention will 
become apparent from the following detailed description of illustrative embodiments 
5 thereof, which is to be read in connection with the accompanying drawings. 

Brief Description of the Drawing s 

FIG. 1 is a diagram illustrating a pervasive computing device projecting a user 

input display and a user output display on separate surfaces, according to an embodiment 

of the present invention; 
10 FIG. 2 is a diagram illustrating a mirror system, a micro projector and the 

resulting split image in the pervasive computing device, according to an embodiment of 

the present invention; 

FIG. 3 is a diagram illustrating a pervasive computing device projecting a user 

input display and key feedback, according to an embodiment of the present invention; and 
15 FIG. 4 is a diagram illustrating a pervasive computing device projecting a user 

input display and a user output display on a single surface in front of the user, according 

to an embodiment of the present invention. 

Detailed Description of Preferred Embodiment 

As will be illustrated in detail below, the present invention introduces apparatus 
20 and techniques for projecting user interfaces for computing devices. A user input display 
may be projected on a first surface while a user output display may be projected on a 
second surface using a single projector of a pervasive computing device. The user input 
display is preferably a virtual keyboard image. By "virtual" keyboard it is meant, a 
projected image on a surface that resembles a keyboard and performs the same functions 
25 as a keyboard with the assistance of a keystroke detection system. While embodiments 
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refer to "pervasive" computing devices, the invention is not limited thereto, and could be 
implemented with other computing devices that would benefit from the use of virtual 
input/output devices. 

Referring initially to FIG. 1 , a diagram illustrates a pervasive computing device 
5 projecting a user input display and a user output display on separate surfaces, according to 
an embodiment of the present invention. A pervasive computing device 101 projects a 
virtual keyboard image 105 on a first surface in front of pervasive computing device 101, 
and between pervasive computing device 101 and the user. A user output display image 
104 is projected behind pervasive computing device 101 onto a flat vertical surface 103. 

10 The first surface is preferably disposed in a plane perpendicular to the second surface. A 
single micro projector 102 is used with a mirror system (not shown but to be described in 
detail in accordance with FIG. 2) to reflect user output display image 104 to the rear and 
project virtual keyboard image 105 in front of the user. A multiple projector system may 
also be used to create the two images on separate surfaces. Virtual keyboard image 105 

15 may also comprise a virtual scratch pad or other pointing device. An example of one of 
several micro projectors is a projection system using the Texas Instruments Digital Micro 
mirror Display (DMD). 

Referring now to FIG. 2, a diagram illustrates a mirror system, a micro projector, 
and the resulting split image in the pervasive computing device, according to an 

20 embodiment of the present invention. It is to be appreciated that the mirror system and 
micro projector may be integral with the computing device. The image from micro 
projector 102 comprises virtual keyboard image 105 in its lower portion 202, and user 
output display image 104 in its upper portion 203. A lower mirror 204 of the mirror 
system catches upper portion 203 and reflects it to an upper mirror 201, which projects 

25 user output display image 104 onto rear surface 103. Lower portion 202 is projected, 
without interference from mirror system 201, 204, in front of pervasive computing device 
101. 
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In operation, lower portion 202 of the projected image with virtual keyboard 
image 105 is superimposed on a keystroke detection system (not shown). This keystroke 
detection system is well known in the art and is described in U.S. Patent No. 6,266,048, 
which is incorporated herein by reference. This system involves laser beams emitted 
5 from pervasive computing device 101 across virtual keyboard image 105, substantially 
parallel to the surface to which virtual keyboard image 105 is projected. When the laser 
beams are interrupted over a virtual key of virtual keyboard image 105 and this 
interruption is detected by a sensor, input is created for processing by the pervasive 
computing device. It is to be appreciated, however, that the invention is not limited to 

1 0 use with any particular keystroke detection system. 

Virtual keyboard image 105 simulates a regular physical keyboard for good 
ergonomics. Upper portion 203 of the projected image is the same as an image that 
would normally be displayed on a computer screen. Upper portion 203 is also in a 
location where a user would look if working with a traditional computer screen. In 

15 addition to the advantage of using only a single projector, the mirror system is also 
designed so that a larger image may be produced even when a short distance exists 
between the pervasive computing device and the vertical display surface. The size of the 
projected image depends on the total distance from micro projector 102 to lower mirror 
203, upper mirror 201 and vertical display surface 103. Because of the folding effect of 

20 the mirror system, the size of this projected image is larger than if the image is projected 
from the micro-projector directly to the display surface. Further enlargement of the 
projected image is possible if the distance between the projector and the mirror system is 
adjustable. 

Referring now to FIG. 3, a diagram illustrates a pervasive computing device 
25 projecting a user input display and keystroke feedback, according to an embodiment of 
the present invention. An additional improvement of the present invention is the addition 
of typing feedback when a key stroke is detected by the keystroke detection system. This 
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feedback may be a configurable audio click or a visual display on either virtual keyboard 
image 105 or user output display 104. Informing the user that a key 106 has been input is 
necessary since the lack of real keys leaves the user uncertain as to whether a key stroke 
was entered into the system. This visual and/or audible feedback permits the user to 
5 quickly recognize whether the desired input was created. Audible feedback may be in the 
form of a click or beep emanating from pervasive device 101. Visual feedback may be 
the changing of the color or the shape of key 106 on virtual keyboard image 105. 
Changing the shape of key 106 may include enlarging the key as shown in FIG. 3. Visual 
feedback may also include displaying an indicating symbol on user output display 104. It 

10 is to be appreciated that the present invention is not limited to the type of mechanism 
used to provide keystroke feedback. 

This feedback technique may be incorporated into pervasive computing device 
101 as a software application. The feedback mechanism may also be implemented by a 
circuit or a controller configurable for providing the functions described. 

15 Referring now to FIG. 4, a' diagram illustrates a pervasive computing device 

projecting a user input display arid a user output display on a single surface in front of the 
user. This embodiment is a front projection system. Advantageously, the configuration 
of the present invention uses a single projector. Virtual keyboard image 105 is projected 
close to the user and user output display image 104 is projected between virtual keyboard 

20 image 105 and pervasive computing device 101. The front projection system may also 
project user output display image 104 on a separate surface and different plane than 
virtual keyboard image 105. For example, user output display image 104 may be 
displayed on a wall behind the user for presentation purposes, while pervasive computing 
device 101 and virtual keyboard image 105 remain in front of the user. In such an 

25 instance, pervasive computing device 101 may be elevated by a tripod so that the user is 
not in the path of projection of output display image 104. A keystroke detection system 
and typing feedback can also be applied with this system. 
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Accordingly, as described herein, the present invention provides apparatus and 
techniques for projecting user interfaces for computing devices such as pervasive 
computing devices and, more particularly, for projecting a user input display and a user 
output display. In one aspect of the invention, the user input display and the user output 
5 display are projected on different surfaces. Other aspects of the invention include the use 
of a single projector and the emission of feedback when a keystroke is detected on the 
virtual keyboard image. The inventive apparatus and techniques for providing a projected 
user interface are applicable to a large number of applications such as PDAs, pocket PCs, 
tablet PCs, and smart camera/PDA phones. 
10 Although illustrative embodiments of the present invention have been described 

herein with reference to the accompanying drawings, it is to be understood that the 
invention is not limited to those precise embodiments, and that various other changes and 
modifications may be made by one skilled in the art without departing from the scope or 
spirit of the invention. 
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